{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Jupyter笔记本中的可重用性问题\n",
    "\n",
    "这个笔记本演示了Jupyter在代码重用方面的困难：\n",
    "1. 难以作为脚本执行\n",
    "2. 难以导入函数和类\n",
    "3. 需要额外工具进行转换"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题1：无法直接作为脚本执行\n",
    "\n",
    "Jupyter笔记本是JSON格式，不能直接用Python解释器运行"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这是一个有用的数据处理函数\n",
    "def clean_data(data):\n",
    "    \"\"\"\n",
    "    清理数据：移除空值，标准化格式\n",
    "    \"\"\"\n",
    "    import pandas as pd\n",
    "    import numpy as np\n",
    "    \n",
    "    if isinstance(data, list):\n",
    "        data = pd.DataFrame(data)\n",
    "    \n",
    "    # 移除空值\n",
    "    data = data.dropna()\n",
    "    \n",
    "    # 标准化数值列\n",
    "    numeric_columns = data.select_dtypes(include=[np.number]).columns\n",
    "    for col in numeric_columns:\n",
    "        data[col] = (data[col] - data[col].mean()) / data[col].std()\n",
    "    \n",
    "    return data\n",
    "\n",
    "print(\"数据清理函数已定义\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 另一个有用的分析函数\n",
    "def analyze_data(data):\n",
    "    \"\"\"\n",
    "    分析数据：计算基本统计信息\n",
    "    \"\"\"\n",
    "    import pandas as pd\n",
    "    \n",
    "    results = {\n",
    "        'shape': data.shape,\n",
    "        'columns': list(data.columns),\n",
    "        'dtypes': data.dtypes.to_dict(),\n",
    "        'summary': data.describe().to_dict()\n",
    "    }\n",
    "    \n",
    "    return results\n",
    "\n",
    "print(\"数据分析函数已定义\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 主要的处理流程\n",
    "def main_pipeline(raw_data):\n",
    "    \"\"\"\n",
    "    完整的数据处理流程\n",
    "    \"\"\"\n",
    "    print(\"开始数据处理流程...\")\n",
    "    \n",
    "    # 清理数据\n",
    "    cleaned_data = clean_data(raw_data)\n",
    "    print(f\"数据清理完成，剩余 {len(cleaned_data)} 行\")\n",
    "    \n",
    "    # 分析数据\n",
    "    analysis_results = analyze_data(cleaned_data)\n",
    "    print(\"数据分析完成\")\n",
    "    \n",
    "    return cleaned_data, analysis_results\n",
    "\n",
    "print(\"主流程函数已定义\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**问题说明**：\n",
    "\n",
    "上面定义了一些有用的函数，但是：\n",
    "\n",
    "1. **无法直接执行**：不能运行 `python notebook.ipynb`\n",
    "2. **需要转换**：必须使用 `jupyter nbconvert --to script` 转换为.py文件\n",
    "3. **转换后的代码混乱**：包含大量注释和格式问题\n",
    "4. **依赖Jupyter**：需要安装Jupyter才能进行转换"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 测试代码\n",
    "if __name__ == \"__main__\":\n",
    "    import pandas as pd\n",
    "    import numpy as np\n",
    "    \n",
    "    # 创建测试数据\n",
    "    test_data = pd.DataFrame({\n",
    "        'A': np.random.randn(100),\n",
    "        'B': np.random.randn(100),\n",
    "        'C': np.random.choice(['X', 'Y', 'Z'], 100)\n",
    "    })\n",
    "    \n",
    "    # 添加一些空值\n",
    "    test_data.loc[::10, 'A'] = np.nan\n",
    "    \n",
    "    # 运行流程\n",
    "    cleaned, results = main_pipeline(test_data)\n",
    "    \n",
    "    print(\"\\n处理结果：\")\n",
    "    print(f\"原始数据形状: {test_data.shape}\")\n",
    "    print(f\"清理后形状: {cleaned.shape}\")\n",
    "    print(f\"数值列数量: {len(results['summary'])}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题2：难以导入和重用\n",
    "\n",
    "其他Python程序无法直接导入Jupyter笔记本中的函数"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这个有用的工具类无法被其他程序直接导入\n",
    "class DataProcessor:\n",
    "    \"\"\"\n",
    "    数据处理工具类\n",
    "    \"\"\"\n",
    "    \n",
    "    def __init__(self, config=None):\n",
    "        self.config = config or {}\n",
    "        self.processed_count = 0\n",
    "    \n",
    "    def process_batch(self, data_batch):\n",
    "        \"\"\"\n",
    "        批量处理数据\n",
    "        \"\"\"\n",
    "        results = []\n",
    "        for item in data_batch:\n",
    "            processed_item = self._process_single(item)\n",
    "            results.append(processed_item)\n",
    "            self.processed_count += 1\n",
    "        \n",
    "        return results\n",
    "    \n",
    "    def _process_single(self, item):\n",
    "        \"\"\"\n",
    "        处理单个数据项\n",
    "        \"\"\"\n",
    "        # 简单的处理逻辑\n",
    "        if isinstance(item, (int, float)):\n",
    "            return item * 2\n",
    "        elif isinstance(item, str):\n",
    "            return item.upper()\n",
    "        else:\n",
    "            return str(item)\n",
    "    \n",
    "    def get_stats(self):\n",
    "        \"\"\"\n",
    "        获取处理统计信息\n",
    "        \"\"\"\n",
    "        return {\n",
    "            'processed_count': self.processed_count,\n",
    "            'config': self.config\n",
    "        }\n",
    "\n",
    "print(\"DataProcessor类已定义\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 测试工具类\n",
    "processor = DataProcessor({'batch_size': 10})\n",
    "\n",
    "test_batch = [1, 2, 'hello', 3.14, 'world', [1, 2, 3]]\n",
    "results = processor.process_batch(test_batch)\n",
    "\n",
    "print(\"处理结果:\", results)\n",
    "print(\"统计信息:\", processor.get_stats())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**问题说明**：\n",
    "\n",
    "如果其他Python程序想要使用 `DataProcessor` 类：\n",
    "\n",
    "```python\n",
    "# 这样做不行！\n",
    "from notebook import DataProcessor  # 错误：无法导入.ipynb文件\n",
    "```\n",
    "\n",
    "需要额外的步骤：\n",
    "1. 手动复制代码到.py文件\n",
    "2. 或者使用特殊的导入工具（如nbimporter）\n",
    "3. 或者转换整个笔记本为脚本"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题3：部署和自动化困难"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# 这个函数可能需要在生产环境中使用\n",
    "def generate_report(data, output_path):\n",
    "    \"\"\"\n",
    "    生成数据报告\n",
    "    \"\"\"\n",
    "    import matplotlib.pyplot as plt\n",
    "    import pandas as pd\n",
    "    from datetime import datetime\n",
    "    \n",
    "    # 创建报告\n",
    "    report = {\n",
    "        'timestamp': datetime.now().isoformat(),\n",
    "        'data_summary': {\n",
    "            'total_rows': len(data),\n",
    "            'columns': list(data.columns),\n",
    "            'memory_usage': data.memory_usage(deep=True).sum()\n",
    "        }\n",
    "    }\n",
    "    \n",
    "    # 生成图表\n",
    "    fig, axes = plt.subplots(2, 2, figsize=(12, 8))\n",
    "    \n",
    "    # 数值列的分布\n",
    "    numeric_cols = data.select_dtypes(include=['number']).columns\n",
    "    if len(numeric_cols) > 0:\n",
    "        data[numeric_cols[0]].hist(ax=axes[0, 0])\n",
    "        axes[0, 0].set_title(f'{numeric_cols[0]} 分布')\n",
    "    \n",
    "    # 保存报告\n",
    "    plt.savefig(f\"{output_path}_charts.png\")\n",
    "    \n",
    "    # 保存数据摘要\n",
    "    import json\n",
    "    with open(f\"{output_path}_summary.json\", 'w') as f:\n",
    "        json.dump(report, f, indent=2)\n",
    "    \n",
    "    print(f\"报告已保存到 {output_path}\")\n",
    "    return report\n",
    "\n",
    "print(\"报告生成函数已定义\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**部署问题**：\n",
    "\n",
    "1. **无法直接调度**：cron作业或任务调度器无法直接运行.ipynb文件\n",
    "2. **容器化困难**：Docker容器中运行Jupyter笔记本需要额外配置\n",
    "3. **CI/CD集成复杂**：持续集成流水线难以处理笔记本格式\n",
    "4. **命令行参数**：难以向笔记本传递命令行参数\n",
    "\n",
    "典型的解决方案需要额外步骤：\n",
    "```bash\n",
    "# 需要先转换\n",
    "jupyter nbconvert --to script notebook.ipynb\n",
    "# 然后才能运行\n",
    "python notebook.py\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 问题4：版本控制和协作困难"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**版本控制问题**：\n",
    "\n",
    "1. **JSON格式复杂**：笔记本是复杂的JSON结构，难以进行有意义的diff\n",
    "2. **输出污染**：执行结果和元数据会污染版本历史\n",
    "3. **合并冲突**：多人协作时容易产生难以解决的合并冲突\n",
    "4. **代码审查困难**：在GitHub等平台上难以进行有效的代码审查\n",
    "\n",
    "示例问题：\n",
    "```json\n",
    "{\n",
    "  \"execution_count\": 42,\n",
    "  \"outputs\": [\n",
    "    {\n",
    "      \"data\": {\n",
    "        \"text/plain\": [\"很长的输出...\"]\n",
    "      }\n",
    "    }\n",
    "  ]\n",
    "}\n",
    "```\n",
    "\n",
    "这些元数据会在每次执行时改变，导致不必要的版本差异。"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 总结：Jupyter的可重用性限制\n",
    "\n",
    "Jupyter笔记本在可重用性方面存在以下主要问题：\n",
    "\n",
    "### 1. 执行限制\n",
    "- 无法直接作为Python脚本运行\n",
    "- 需要Jupyter环境才能执行\n",
    "- 转换过程复杂且容易出错\n",
    "\n",
    "### 2. 导入限制\n",
    "- 其他Python程序无法直接导入笔记本中的函数和类\n",
    "- 需要特殊工具或手动转换\n",
    "- 破坏了Python的标准模块系统\n",
    "\n",
    "### 3. 部署限制\n",
    "- 生产环境部署复杂\n",
    "- 自动化和调度困难\n",
    "- 容器化需要额外配置\n",
    "\n",
    "### 4. 协作限制\n",
    "- 版本控制困难\n",
    "- 代码审查体验差\n",
    "- 团队协作效率低\n",
    "\n",
    "这些问题使得Jupyter笔记本更适合探索性分析和原型开发，而不适合构建可重用的生产系统。"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "marimo-ui (3.12.11)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.12.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
